Optimal gradient tracking for decentralized optimization

In this paper, we focus on solving the decentralized optimization problem of minimizing sum n objective functions over a multi-agent network. The agents are embedded in an undirected graph where they can only send/receive information directly to/from their immediate neighbors. Assuming smooth and strongly convex functions, propose Optimal Gradient Tracking (OGT) method that achieves optimal gradient computation complexity $$O\left( \sqrt{\kappa }\log \frac{1}{\epsilon } \right) $$ communication \sqrt{\frac{\kappa }{\theta }}\log simultaneously, $$\kappa $$\frac{1}{\theta }$$ denote condition numbers related to graph, respectively. To our best knowledge, $$\textsc {OGT}$$ is first single-loop gradient-type both complexities. development involves two building blocks also independent interest. one another new tracking termed “Snapshot” (SS-GT), which complexities \frac{\sqrt{\kappa }}{\theta , {SS-GT}$$ be potentially extended more general settings compared OGT. second technique Loopless Chebyshev Acceleration (LCA), implemented “looplessly” but similar effect by adding multiple inner loops acceleration algorithm. addition LCA accelerate many other based methods with respect number .